In Deep reinforcement Learning, experience replay buffers are used to reduce the effects of sequential data and make better use of past experiences. Prioritized Experience Replay (PER) improves upon random sampling by selecting transitions based on their temporal difference (TD) error. However, PER does not consider how important each transition is or how many times it has been used during training. In this paper, we propose a new method for adaptive prioritization that takes into account three additional transition-level factors: reward, usage count (counter), and policy probability—collectively referred to as RCP values. These values are normalized and used alongside the TD error to calculate the probability of selecting each transition from the replay buffer. We evaluate our method on several Atari environments and show that using any of the RCP values individually can improve performance compared to standard PER. To combine all three RCP components, we explore three aggregation functions: minimum, maximum, and mean. Experimental results show that the best aggregation method depends on the environment. However, the mean function generally provides stable improvements across tasks, as it balances all RCP signals and avoids over-relying on any single factor.

A social network consists of individuals and the relationships between them, which often influence each other. This influence can propagate behaviors or ideas through the network, a phenomenon known as influence propagation. This concept is crucial in applications like advertising, marketing, and public health. The influence maximization (IM) problem aims to identify key individuals in a social network who, when influenced, can maximize the spread of a behavior or idea. Given the NP-hard nature of IM, non-exact algorithms, especially metaheuristics, are commonly used. However, traditional metaheuristics like the variable neighborhood search (VNS) struggle with large networks due to vast solution spaces. This paper introduces DQVNS (Deep Q-Learning Variable Neighborhood Search), which integrates VNS with Deep reinforcement Learning (DRL) to enhance neighborhood structure determination in VNS. By using DQVNS, we aim to achieve performance similar to population-based algorithms and utilize the information created step by step during the algorithm's execution. This adaptive approach helps the VNS algorithm choose the most suitable neighborhood structure for each situation and find better solutions for the IM problem. Our method significantly outperforms existing metaheuristics and IM-specific algorithms. DQVNS achieves a 63% improvement over population-based algorithms on various datasets. The results of implementation on different real-world social networks of varying sizes demonstrate the superiority of this algorithm compared to existing metaheuristic, IM-specific algorithms, and network-specific measures.

The Laplace transform is widely used in science and technology to deal with complex problemsin stability and control systems. The modified Laplace transform has been applied in physics andmathematics to solve boundary layer equations in ordinary differential equations with variablecoefficients. The q-calculus appeared as a connection between mathematics and physics. It hasmany applications in different mathematical areas, such as number theory, combinatory theory,orthogonal polynomials, essential hyper-geometric functions, quantum mechanics, and relativity.Laplace transform, and its several extended versions are used frequently. The Double Laplacetransform applies to solving some q-functional and partial q-differential equations. Q-calculushas been used to solve complex and more potentially typical problems in a larger domain toinvestigate the calculus without limits for getting more generalizations. In the paper, weintroduce the Double-modified Laplace transform in q-calculus, namely the q-Double modifiedLaplace transform, and establish some properties. Furthermore, several propositions concernedwith q-Double modified Laplace transform are explored.

Access to cash for many in society is remainingessential during the current COVID-19 lock-down around theglobe. A smart city requires the banking industry to exploit IoTand Artificial Intelligence (AI) in order to track its ATM networkand predict outages due to cash shortages. In this paper, we studythe real-time cash replenishment planning problem under outflowuncertainty where the fee of the security companies grows if thereplenishment ends up falling on a weekends/holidays. Our modelis based by the Double Deep Q-Network (DQN) algorithm whichcombines popular Q-Learning with a Deep neural network. Theproposed method is used to minimize the ATM replenishmentcost where the cash demand changes dynamically at each day.The performance analysis of the proposed method for differentamounts of replenishment cash shows that the the proposedmethod can effectively work under real word conditions andreduce the ATM operational cost compared with the other stateof-the-art cash demand prediction schemes.Index Terms—cash replenishment planning, Deep Learning,ATM, reinforcement Learning, Double Q-network.

Abstract - - Abstract - - Abstract - This paper proposes a master-slave approach to simultaneously control two drones with the aim of carrying an object toward a goal. The proposed method utilizes the Double Deep Q-Learning (DDQN) technique to train a master agent to be able to carry an object toward a goal with the help of a slave agent. This procedure is implemented such that the master agent gathers the observations and specifies the actions to be made by itself and the slave agent. Indeed, the slave agent just applies a predefined action and does not process any input for producing the output. This manner of Learning, leads to a unified convergence to an optimal solution compared to the situation in which each agent is trained separately. To verify the functionality of the proposed method, the algorithm is examined in the webots simulation environment. The simulations show that the introduced method has a good performance when controlling the drones to reach to the goal. The introduced method, other than algorithmic benefits which leads to a faster convergence of the model, suggests some reduction in the processing demand. The reason is that the Learning procedure is guided by one of the agents and consequently only one of the agents is responsible for doing the calculations that lead to choosing the action. In this scenario, the slave agent does not require any processing resources for choosing the action and just simply applies a predefined action dictated by the master agent.